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ABSTRACT , .... 

The system is a comprehensive set of computer 
programs and procedures for assisting in the preparation, scoring and 
analysis of multiple choice test batteries. The system may be used 
nith either standardized or locally developed tests. For the upper 
grades the system accepts pupil responses on mark-sense ansver 
sheets. For Joner grades pupil responses may be keypunched. Nhen 
Bark->sense forms are used, the system can also interrogate the school 
dijgtricts pupil files and preprint ansver sheets for the pupils who 
are to Ije tested. A number of tests for each pupil may be analyzed at 
the saie time. A score is computed for each test and a pupil's 
performance on each test relative to all pupils taking the test in 
the district is reported. Various types of norms can be used with the 
system. Mean scores and other summary statistics are reported for 
groups of pupils. All computer programs are written in American 
National Standard COBOL to facilitate implementation on most 
computers. The entire system is designed and written in a modular 
fashion to permit modifications to existing reports or additional 
reports to be added. This design permits the system to be understood 
by the computer analyst. For related document see TM003447. [ This 
document has been reproduced from the best copy available.] (EE) 
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Overview 

— The-Racine Feedback and Diagnostlc Systera :1s ^^^^^ 
computer programs and procedures for assisttng in the preparation, scoring 
and analysis of multiple choice test batteries. The system may be used with 
either standardized tests (such as ll^e Stanford Achievejnent Tests) or with 
diagnostic tests prepared by teachers or curriculum consultants within the 
district. For the upper grades cho system accepts pupil responses on mark 
sense answer sheets. For lower grades pupil responses may be keypunched. 
When mark -sense forms are used, the system can also interrogate the school 
districts? pupil files and preprint answer sheets for the pupils who are to be 
tested. 

All other input to the system is provided through common format input 
coding sheets that are easy to use and require a minimum amount of coding to 
be done for each test. 

A number of tests for each pupil may be analyzed at the same time. All of 
the tests together comprise a test battery. A score is computed for each test 
and a pupiVs performance on each test relative to all pupils taking the test 
in the district is reported. If national norms (or other norm data) are 
ava "liable for the test, this data may be provided as input and the performance 
(grade equivalent, percentile and stanine) of each pupil relative to the norm 
data is reported. Norms that are dependent on the age of the pupil (commonly 
used with IQ tests) may be used also. 

A number of statistics are reported for tests (or scoring sets) and the 
items within them. The system also permits items to be grouped into other 
categories called item sets. 
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Item sets are a means of grouping items into skill areas that are not 
necessarily -the- same-groupings us€d for~t Among 
other uses of item sets, the system Itsts the pupils in each room who do not 
answer correctly at least a specified percent of items in each item set* 

Mean scores and other summary statistics are reported for groups of 
pupil s« Typically these groups are rooms (and rooms are used in the 
description below). Other groupings are possible however. Examples are. 
groups such as "girls in room 103" or "pupils with special needs in 
Westview ichool". 

All computer programs are written in American National Standard COBOL 
(ANS COBOL) to facilitate implementation on most computers. The system will 
operate on a 32K byte, machine. Expandable tables in the programs and other 
features^ however, permit taking advantage of larger machines and reducing 
-the required running time. A non- technical Users Guide and a more technical 
System Documentation Manual are available for the system. The Users Guide 
permits a non-technical user to prepare all required input for the system and 
to coordinate the test preparation, administration and analysis functions. 
The System Documentation Manual is designed for the systems analyst or 
computer programmer who is to maintain the computer programs and possibly 
modify them. 

The entire system is designed and written in a modular fashion to permit 
modifications to existing reports or additional reports to be added. This 
design also permits the system to be understood by the computer analyst with 



a minimum amount of analysis and effort. 
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Reports Avai 1 abl e from the Sys tem 

A primary goal in the development of the system was to make the testing 
process a valuable learning experience for the student and the teacher. The 
reports prepared by the system reflect this goal in the amount of infonation 
they provide to the student and the teacher in addition to the test analyst 
or coordinator. Rather than simply giving the student a scores the system 
has the capability of telling the student why his incorrect choices were wrong 
and providing assistance in strengthening the students skills irv his weak ones. 
Other reports enable the teacher to identify areas in which the entire class 
needs additional work and provide assistance in prescribing additional 
instructional material. 

A variety of reports are available wfth different information and varying 
amounts of detail for the several users for whom the reports are prepared. 
All reports are optional and are prepared only if requested. 

O Figure 1 shows the Individual Pupil Prescriptions for incorrect responses. 
This report provides valuable feedback to the pupil by including a prescription 
for each incorrect response. These prescriptions may indicate why the response 
was incorrect or may direct the pupil to additional instructional material. 

O A summary of pupil scores for all pupils in each room is given on the 
report shown in Figure 2. Each pupils performance relative to all test takers 
in the district and relative to national norms is reported for each test taken. 

O The Percentile report shown in Figure 3 provides the frequency of 

_„0£currance,„the„percenti_le_score„and_the_stanine_for--each-^^ This 

report is produced for each room, for each school in the district arid for the 
entire district. The report may also be prepared for any other grouping of 
rooms e.g., rooms using teacher aids, rooms using team teaching, inner city 
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Figure 1 
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Figure 2 
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Figure 3 
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schools, etc. The report sFiown describes raw scores. When standardized norm 
data is provided for the test, this report may be reque.sted for standardized 
scores also. 

O The percent of pupils selecting each response for each item is given In the 
report Illustrated in Figure 4. This report can be an aid in determining the 
areas in which pupils needs additional work. As with the Percentile report, 
this report is prepared for each room, for each school in the district, for 
the entire district and for any other specified room groupings. The report 
shown in the Figure shows items grouped by scoring set. The report is also 
available with items grouped by item set. 
O The Item Set Response Analysis Report, illustrated in Figure 5 is another 
aid in identifying instructional areas where pupils in a room need additional 
work. This report is prepared for each room with items grouped by item set. 
An accompanying sheet (shown at the bottom of the Figure) identifies the names 
of the item sets and test parts. 

O The Item Set Fail/Marginal report shown in Figure 6 is particularly 
useful in larger districts where curriculum consultants are uti lized to - 
develop instructional materials and assist in overall curriculum planning. 
This report, which is prepared for each room, identifies for each item set 
the pupils who either failed or performed marginally on the item set. 
("Failing" an item set is not answering correctly at least a speicifed 
percent of items comprising the item set. A "marginal" performance is not 
answering correctly a higher specified percent of the items in the set.) 
Remedial prescriptions are provided for use by the teacher to help improve 
the skills of the pupils who failed or performed marginally. 
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Figure U 
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Figure 4 
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Figure 5 
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O The FMl /Marginal suranary report which is shown in Figure 7 assists the 
curriculum consultants or other personnel at the school or district level • 
This report Indlcate^i; the number of pupils taking each Item set In each room 
and the number and percent of pupils In each room who failed or performed 
marginally on the Item set* 

A number of additional reports are also prepared that assist the test 
analyst to Insure that the input to the system has bee?i prepared and 
processed correctly. Examples of these are Included in the "Input" section 
below. 

Labels 

Adhesive labels for inclusion in the pupils permanent records are also 
produced by the system. These labels show both district and nationally 
standardized test performance. Sample labels are shown in Figure 8. 

Input to the System 

As mentioned above, pupil responses are entered in the system either 
through mark sense answer sheets or on keypunched "IBM" cards. Othe*" required 
input is the description of the test battery needed to produce the reports 
requested. This information is first entered on input coding sheets as 
illustrated in Figures 9 and 10. These sheets are then keypunched and used as 
input to the system. Substantial editing is performed on this, input to 
insure its accuracy. Various reports are also produced to assist the test 
analyst In determining that the input data has been prepared correctly. Two 
of these '*input validation" reports are shown in Figures 11 and 12. 
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Figure 8 



SikMFLE USELS> 



HAAKENSON MICHAEL 14930 
FALL 1972 TESTTNG 

LOCAL — NATL— — 
R.S PC ST G.E PC ST 
LANG ARTS 01 27 / 4 LOCAL ONLY 
MATH SKILL 10 83 / 7 508 01 / 1 
READING 01 14 / 3 LOCAL ONLY 

STEP SOCIA 21 90 / 8 427 13/3 



KAPLAN DANIEL OTTO 



37632 



FALL 1972 TESTING 



LANG ARTS 
MATH SKILL 
READING 
STEP SOCIA 



LOCAL — NATL— 
R.S PC ST G.E PC ST 

02 67/6 LOCAL ONLY 
08 66 / 6 505 Oi / I 

03 69 / 6 LOCAL ONLY 
25 98 / 9 433 25/4 



CUDWIG RAYMOND JUOE 45816 
FALL 1972 TESTING 

LOCAL ~NATL— — 
R.S PC ST G.E PC ST 

00 00 / 1 LOCAL ONLY 
05 25 / 4 504 01 / 1 

01 14 / 3 LOCAL ONLY 
08 05 / ^ 405 01/1 



LANG ARTS 
MATH SKILL 
READING 
STEP SOCIA 
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Figure 10 
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Figure 11 
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Figure 12 
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Much of the input data needed to be coded and keypunched only once for a 
district (and then just updated when personnel changes, etc. are made). The 
same input cards may then be used for a number of different tests. 

Implementation of the System 

As mentioned above, the entire system is written in ANS COBOL to expedite 
implementation. Interfaces with existing district files and systems are 
required only for preprinting mark sense answer sheets and for looking up * 
pupil identification data (i.e., name and birthdate) when the completed 
answer sheet (or keypunched cards) are being scored. If the school district 
is not currently meitaining these files in a computerized form they can 
easily be developed for use by this system. 

Because of the "self- contained" nature of the system it may be 
Implemented on a "computer utility" or some other computer not managed by 
the school district. 

Implementation of the system may be accomplished through use of its^^^o 
instructional development and data processing personnel. Implementation 
assistance for both the computer system and for the development of meaningful 
instructional feedback is available, however. For further Information 
corrcerning any aspect of the system please contact 
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